Evaluation of Explanation Methods of AI -- CNNs in Image Classification Tasks with Reference-based and No-reference Metrics

The most popular methods in AI-machine learning paradigm are mainly black boxes. This is why explanation of AI decisions is of emergency. Although dedicated explanation tools have been massively developed, the evaluation of their quality remains an open research question. In this paper, we generalize the methodologies of evaluation of post-hoc explainers of CNNs' decisions in visual classification tasks with reference and no-reference based metrics. We apply them on our previously developed explainers (FEM, MLFEM), and popular Grad-CAM. The reference-based metrics are Pearson correlation coefficient and Similarity computed between the explanation map and its ground truth represented by a Gaze Fixation Density Map obtained with a psycho-visual experiment. As a no-reference metric, we use stability metric, proposed by Alvarez-Melis and Jaakkola. We study its behaviour, consensus with reference-based metrics and show that in case of several kinds of degradation on input images, this metric is in agreement with reference-based ones. Therefore, it can be used for evaluation of the quality of explainers when the ground truth is not available.Comment: Due to a bug found in the code, all tables and figures were redone. The new results did not change the main conclusion, except for the best explainer. FEM has performed better than MLFEM; 25 pages, 16 tables, 16 figures; Submitted to "Advances in Artificial Intelligence and Machine Learning" (ISSN: 2582-9793

Accelerating, hyperaccelerating, and decelerating networks

Many growing networks possess accelerating statistics where the number of links added with each new node is an increasing function of network size so the total number of links increases faster than linearly with network size. In particular, biological networks can display a quadratic growth in regulator number with genome size even while remaining sparsely connected. These features are mutually incompatible in standard treatments of network theory which typically require that every new network node possesses at least one connection. To model sparsely connected networks, we generalize existing approaches and add each new node with a probabilistic number of links to generate either accelerating, hyperaccelerating, or even decelerating network statistics in different regimes. Under preferential attachment for example, slowly accelerating networks display stationary scale-free statistics relatively independent of network size while more rapidly accelerating networks display a transition from scale-free to exponential statistics with network growth. Such transitions explain, for instance, the evolutionary record of single-celled organisms which display strict size and complexity limits

Investigation of the 6He cluster structures

The 4He+2n and t+t clustering of the 6He ground state were investigated by means of the transfer reaction 6He(p,t)4He at 25 MeV/nucleon. The experiment was performed in inverse kinematics at GANIL with the SPEG spectrometer coupled to the MUST array. Experimental data for the transfer reaction were analyzed by a DWBA calculation including the two neutrons and the triton transfer. The couplings to the 6He --> 4He + 2n breakup channels were taken into account with a polarization potential deduced from a coupled-discretized-continuum channels analysis of the 6He+1H elastic scattering measured at the same time. The influence on the calculations of the 4He+t exit potential and of the triton sequential transfer is discussed. The final calculation gives a spectroscopic factor close to one for the 4He+2n configuration as expected. The spectroscopic factor obtained for the t+t configuration is much smaller than the theoretical predictions.Comment: 10 pages, 11 figures, accepted in PR

Predictive biometrics: A review and analysis of predicting personal characteristics from biometric data

Interest in the exploitation of soft biometrics information has continued to develop over the last decade or so. In comparison with traditional biometrics, which focuses principally on person identification, the idea of soft biometrics processing is to study the utilisation of more general information regarding a system user, which is not necessarily unique. There are increasing indications that this type of data will have great value in providing complementary information for user authentication. However, the authors have also seen a growing interest in broadening the predictive capabilities of biometric data, encompassing both easily definable characteristics such as subject age and, most recently, `higher level' characteristics such as emotional or mental states. This study will present a selective review of the predictive capabilities, in the widest sense, of biometric data processing, providing an analysis of the key issues still adequately to be addressed if this concept of predictive biometrics is to be fully exploited in the future

Magnetic states and spin-glass properties in Bi0.67Ca0.33MnO3: macroscopic ac measurements and neutron scattering

We report on the magnetic properties of the manganite Bi_{1-x}Ca_{x}MnO_3 (x=0.33) at low temperature. The analysis of the field expansion of the ac susceptibility and the observation of aging properties make clear that a spin glass phase appears below T = 39K, in the presence of magnetic order. Neutron scattering shows both magnetic Bragg scattering and magnetic diffusion at small angles, and confirms this coexistence. In contrast to Pr_{1-x}Ca_{x}MnO_3 (x=0.3-0.33) which exhibits a mesoscopic phase separation responsible for a field driven percolation, the glassy and short range ferromagnetic order observed here does not cause colossal magnetoresistance (CMR).Comment: accepted in Phys Rev

The Biomarker Toolkit - an evidence-based guideline to predict cancer biomarker success and guide development

BACKGROUND: An increased number of resources are allocated on cancer biomarker discovery, but very few of these biomarkers are clinically adopted. To bridge the gap between Biomarker discovery and clinical use, we aim to generate the Biomarker Toolkit, a tool designed to identify clinically promising biomarkers and promote successful biomarker translation. METHODS: All features associated with a clinically useful biomarker were identified using mixed-methodology, including systematic literature search, semi-structured interviews, and an online two-stage Delphi-Survey. Validation of the checklist was achieved by independent systematic literature searches using keywords/subheadings related to clinically and non-clinically utilised breast and colorectal cancer biomarkers. Composite aggregated scores were generated for each selected publication based on the presence/absence of an attribute listed in the Biomarker Toolkit checklist. RESULTS: Systematic literature search identified 129 attributes associated with a clinically useful biomarker. These were grouped in four main categories including: rationale, clinical utility, analytical validity, and clinical validity. This checklist was subsequently developed using semi-structured interviews with biomarker experts (n=34); and 88.23% agreement was achieved regarding the identified attributes, via the Delphi survey (consensus level:75%, n=51). Quantitative validation was completed using clinically and non-clinically implemented breast and colorectal cancer biomarkers. Cox-regression analysis suggested that total score is a significant driver of biomarker success in both cancer types (BC: p>0.0001, 95.0% CI: 0.869-0.935, CRC: p>0.0001, 95.0% CI: 0.918-0.954). CONCLUSIONS: This novel study generated a validated checklist with literature-reported attributes linked with successful biomarker implementation. Ultimately, the application of this toolkit can be used to detect biomarkers with the highest clinical potential and shape how biomarker studies are designed/performed